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ABSTRACT 

Recent publication of numerous books and papers indicates 
the growing importance of Software Quality Metrics [1]. Studies 
at the Boeing Aerospace Company 12,3] have extended this field to 
cover Distributed Computer Systems. Emphasis is placed on 
studying Embedded computer systems, and on viewing them within 
a system life cycle [4]. The approach of J. A. McCall, et.al. 
[5,6], at General Electric was pursued and extended, maintaining 
the hierarchy of quality factors, criteria, and metrics [fig.l]. 
New software quality factors have been added, including Sur- 
vivability, Expandability, and Evolvability [fig. 2]. 

- _ _key;wori)s _ 

Software, Quality, Metrics, Distributed, Survivability, Life Cy- 
cle, Expandability, Evolvability, Virtuality 

INTRODUCTION 

What is a distributed computer system? Enslow [7] requires 
such a system to meet five criteria, while LeLann [8] requires it 
to be a collection of entities participating in system perfor- 
mance. Mauchley and Eckert built the first distributed computer, 
BINAC, circa 1947 » and acknowledged [9] that the structure of the 
human brain, with its two cerebral hemispheres, was ,a guiding 
design metaphor. Dr. Roger Sperry's Nobel Prize in Medicine was 
for experiments performed at Caltech which established that the 
human brain is a distributed computer [10]. We consider a dis- 
tributed system to be formed by the interconnection of potential- 
ly autonomous systems to accomplish system functions cooperative- 
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ly . 


There are several ways the term "distributed" may be inter- 
preted. Data may be distributed, processors may be distributed, 
processes may be distributed, users may be distributed, communi- 
cations may link geographically dispersed clusters of components, 
or some combination of these strategies may be imposed on system 
architecture. Each of these types of distributedness leads to 
design tradeoffs, and to qualitative distinctions between cen- 
tralized and distributed systems. No single model allows 
analysis of all such tradeoffs; data is either specialized, anec- 
dotal, or condensed to "lessons learned" or scenario form. The 
application of Software Quality Metrics should help to provide a 
unifying framework for all such distributed systems. As Norber 
Weiner first emphasized [11], it is possible to build a reliable 
system out of unreliable parts. 

It will be increasingly important to understand distributed 
computer systems. Some of their characteristics will emerge more 
extensively in future configurations. One characteristic peculiar 
to distributed systems, and of importance in the^ 80's, is Geo- 
graphic Dispersion". The extent to which computers within a dis- 
tributed system can be physically displaced from each other, 
range from the centimeter to the mul t 1- t housand-ki 1 ome t e r . Com- 
puters will indeed be "tightly-coupled" over intercontinental 
distances by fiber-optics technology currently under research. 
This technology complements that of the communications satellite. 
Interconnection of even a very small percentage of available com- 
puters will be able to form distributed systems of complexity 
beyond those of today, since by 1999 there will be on the order 
of one billion computers in the world [13]. 

QUALITY METRICS APPROACH 

The approach chosen to evaluate distributed systems is the 
Software Quality Metrics methodology, which has been fruitfully 
applied to the study of a broad range of uniprocessor computers 
and embedded computer systems [1]. Since the 1970's, additional 
factors have been Judged necessary in evaluating the performance 
of software and systems besides that of classic Reliability which 
was a factor closely identified with software and system quality. 
McCall and others [5,6] identified eleven software quality fact- 
ors and developed a system of metrics to predict and assess the 
degree of presence of these factors. As shown in flg.l, each fa- 
ctor is composed of a number of criteria which are further broken 
down into quantitative metrics. The eleven Factors identified : 
Correctness, Reliability, Efficiency, Integrity, Usability, Main- 
tainability, Testability, Flexability, Portability, Reusability, 
and Interoperability. The extension of this approach to distrib- 
uted systems was introduced at last year's workshop by Robert W. 
Lawler of Boeing Aerospace Company [15]. The research conducted 
during the past year, as reported to RADC[2], has concentrated on 
identifying unique characteristics of distributed systems, and on 
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definition or redefinition of factors and criteria which can mea- 
sure these characteristics. Three new software factors, four new 
system factors, twelve new software criteria , and two new system 
criteria have been described, and the factor of Testability has 
been generalized into the factor of Verifiability. Examples of 
these new factors and criteria are described below and in fig. 2. 

DISTRIBUTED SYSTEM CHARACTERISTICS 

How do we approach the identification of the characteristics 
of distributed systems? Distributed System characteristics are 
identified and classified, along with rationales for the 
selection of Distributed Systems. 58 rationales are grouped into 
9 reasons in fig. 3 . The rationales given for selection of a 
distributed system over a uniprocessor system indicate the 
characteristics which people imagine distributed systems, as a 
whole, exhibit. No one system meets more than a fraction of 
these identifications, just as no system life cycle for a distri- 
buted system quite fits into the system life cycle models for 
uniprocessor systems. Instead, we find the distributed system to 
be distributed through time in a distributed life cycle of con- 
current phases of Operation, Revision, and Transition [fig. 4]. 

NEW QUALITY FACTORS 

The main difference between software metrics for a distri- 
buted system and software metrics for a uniprocessor system is 
that the quality of software in a distributed environment depends 
upon the design and performance characteristics of the entire 
system. We therefore distinguish between Software Quality Factors 
and System Quality Factors, although these have impact upon each 
other. The quality factor of Survivability, for example, re- 
flects system performance when one or more nodes or communication 
links become totally nonoper a t ional . The concepts of Reliability 
and Redundancy in a uniprocessor are not broad enough to describe 
Survivability . — — , 

Survivability is a factor which measures the capability of a 
system to operate when one or more components are destroyed. For 
a non- distributed system. Survivability is not a very meaning- 
ful measure. A single unit computer, depending on the degree of 
hardening and the damage received in the tactical environment, 
will usually either continue to operate, or else be completely 
incapacitated. For a geographically dispersed system, it is 
desirable that damage or destruction of individual components 
shall allow the system to continue functioning, albeit at a some- 
what lower level of performance. Survivability, then, might 
measure the likelihood of a distributed system to exhibit this 
"graceful degradation". The 5 criteria within the system quality 
factor of . Surv Ivab il i ty are Autonomy, Distributedness, Anomaly 
Management, Modularity, and Reconfigurability. (See fig. 2) 

Distributed Systems also require metrics to evaluate the capaci- 
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ty of expanding and upgrading the system, so we have identified 
and defined the corresponding factors of Expandability and 
Evolveab il ity . Expandability is the extent to which the system 
capability can be expanded to enhance current functions or to add 
new functions. The criteria within the factor of Expandability 
include; Virtuality, Generality, Modularity, Augment ab il i ty , 
Clarity, Specificity, and Simplicity. Evolvability is the extent 
to which the system performance could be enhanced by the incor- 
poration of new technology. Criteria within Evolvability are 
Virtuality, Generality, Modularity, Clarity, Specificity, and 
Simplicity. In addition, we have defined four new system quality 
factors. Availability, Safety, Transportability, and Interchange- 
ability. 


NEW CRITERIA 

Twelve new software criteria were identified during investi- 
gation of characteristics for distributed systems [2]. These 
criteria are: Compliance, Validity, Clarity, Specificity, Virtu- 
ality, Comprehensibility, Reconfigurability, Distributedness, Au- 
tonomy, Suppor tab il ity , Augmentab illty , and Compatibility 
[fig. 5]. In addition, two new system criteria were identified : 
Self-containedness (an attribute of Transportability) and Homo- 
geneity (an attribute of Interchangeability). A majority of these 
system and software criteria are applicable to uniprocessors as 
well. The following brief discussion on one of the new software 
criteria. Virtuality, shows how the entire system, including the 
human users, needs to be measured to evaluate the system quality. 

For Distributed Systems, there is a new criterion within the 
quality factor for Usability. We refer to this criterion as Vir- 
tuality. The structure of a distributed system can be quite com- 
plex, and it is not always desirable for the user to be appraised 
of this structure. The user may perceive the system in terms of 
a virtual architecture, and be shielded from knowing the actual 
internal representation and location of data. 

Virtuality is a measure of the extent to which the system 
appears to the user as it is intended to appear to the user. The 
user (or users) of a system is not expected or intended to see 
the system's logical, topological, or physical structure. In- 
stead, an abstract "virtual" system is designed. The "real" sys- 
tem supports, emulates, and embodies the designed appearance and 
"feel" of the virtual system. 

Theodor H. Nelson [12] explains the relationship between 
Virtuality and other criteria such as Conceptual Simplicity, 
Machine Independence, and Network File Availability. "Our ap- 
proach to computer design we call 'the design of virtuality.' By 
virtuality we mean the seeming of an object or system, its con- 
ceptual structure, its atmospherics and its feel.... What counts 
is effects, not techniques.... The design of an interactive com- 
puter environment, similarly, should not be based on particular 
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hardware, or a particular display device, or a programming tech- 
nique.... the systems analysis for an interactive system should 
deal with the mental space of the user's experience." 

Virtuality also measures the subjective component of the 
user interface. In the special case of flight training simula- 
tors [14], the "feel" of the system has long been regarded as 
crucial to Usability. "Feel" is evaluated by expert pilots (su- 
perusers). This goes beyond Human Engineering, which concen- 
trates on one display /sensory modality at a time, or on total 
bits per second. "Feel", and ther ef o re Vir t ual i ty , involves ges- 
talt perception, with an emphasis on right-brain holistic activi- 
ty. Virtuality, and the human brain, cannot be ignored when 
studying distributed systems. 

NEW METRICS 

During the next year of this research effort there will be a 
set of metrics developed within the criteria and factors discuss- 
ed above. The existing metrics [6] will be added to, deleted, and 
modified in accordance with results to date. The work yet to be 
performed may be summarized as follows: 

(1) Select Quality Metrics for Validation (Identify those metrics 
that will make the greatest contribution to validating the quali- 
ty measurement framework previously developed); 

(2) Develop Scenarios and Collect Data (Design the data collection 
methodologies and gather relevant data from Boeing Aerospace Com- 
pany projects which use distributed embedded computer systems); 

(3) Validate Metrics (Validation techniques consistent in concept 
and methodology with McCall, et.al. [6], but with multivariate 
regression analysis and other numerical analysis and correlation 
methods; conduct interviews with engineering and management 
personnel to supplement empirical data) ; 

(4) Produce a Report and Handbook (Final Report to be published 
by RADC. A Handbook will be prepared that describes the step- 
by-step procedures required to implement the quality meas- 
urements for distributed systems). 

SUMMARY 

Software Quality Metrics may be applied to the evaluation of 
distributed computer systems. Exactly what constitutes a distrib- 
uted system is disputed in the literature. They have been built 
in various configurations for thirty years, but the human brain 
shares some of the characteristics of these systems and provides 
a valuable model. The approach of McCall et.al., with factors, 
criteria, and metrics, has been extended. New factors and new 
criteria have been defined. New metrics will be devised and val- 
idated as the research described in this paper is continued. 
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USER-ORIENTED VIEW 
OF PRODUCT aUALITY 


SOFTWARE-ORIENTED 
ATTRIBUTES WHICH 
INDICATE aUALITY 


aUANTITATIVE MEASURES 
OF ATTRIBUTES 


Figure 1 Software Ouatity Model 
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Figure Z Relationship of Criteria to 
Software Quality Factors 





{ effectiveness! 





J.Post 
Boeing 
10 of 15 


USABILITY 


VIRTUALITY OPERABILITY I COMMUNICATIVENESS 1 1 VISIBILITY I COMPREHENSIBILITY 1 I TRAINING 




* = New 

** = Different Figure 2 Relationship of Criteria to Software Buality Factors 
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Figure 3 Relationship Between Reasons, Rationales, and 
System Quality Factors (page 1 of 2) 
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Figure 3 Relationship Between Reasons, Rationales, and 
System Quality Factors (page 2 of 2) 
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Figure 4 Quality Life Cycle Scheme 
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Figure 5 Software Quality Criteria Definitions 


r w 

o * 


"*» 3 «« 


CRITERION 

DEFINITION 

•TRAINING 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE TRANSITION FROM CURRENT 
OPERATION OR PROVIDE INITIAL FAMILIARIZATION. 

•COMMUNICATIVENESS 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE USEFUL INPUTS AND OUTPUTS 
WHICH. CAN BE ASSIMILATED. 

• OPERABILITY 

•THOSE ATTRIBUTES OF THE SOFTWARE, WHICH DETERMINE OPERATIONS AND 
PROCEDURES CONCERNED WITH THE OPERATION OF THE SOFTWARE. 

• MODULARITY 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE A STRUCTURE OF HIGHLY 
COHESIVE MODULES WITH OPTIMUM COUPLING. 

• RECONFIGURABILITY* 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE FOR CONTINUITY OF SYSTEM 
OPERATION WHEN ONE OR MORE PROCESSORS* STORAGE UNITS* OR COMMUNICATIONS 
LINKS FAIL. 

• DISTRIBUTEDNESS* 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH DETERMINE THE DEGREE TO WHICH 
SOFTWARE FUNCTIONS ARE GEOGRAPHICALLY OR LOGICALLY SEPARATED WITHIN THE 
SYSTEM. 

•AUTONOMY* 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH DETERMINE ITS DEPENDENCY ON 
INTERFACES. 

•CONCISENESS 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE FOR IMPLEMENTATION OF A 
FUNCTION WITH A MINIMUM AMOUNT OF CODE. 

• SUPPORTABILITY* 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE FOR EASE IN CREATION OF NEW 
SOFTWARE VERSIONS (e.fi.* USE OF HOL. VERSION UPDATE SCHEME). 

• SELF-DESCRIPTIVENESS 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE EXPLANATION OF THE 
IMPLEMENTATION OF A FUNCTION. 

•GENERALITY 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE BREADTH TO THE FUNCTIONS 
PERFORMED. 

• INDEPENDENCE** 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH DETERMINE ITS DEPENDENCY ON THE 
SOFTWARE ENVIRONMENT (COMPUTING SYSTEM* OPERATING SYSTEM. UTILITIES. 
INPUT/OUTPUT ROUTINES. LIBRARIES). 

• AUGMENTABILITY* 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE EXPANSION CAPABILITY FOR 
FUNCTIONS AND DATA. 

•COMPATIBILITY* 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE INTERFACE PROTOCOLS AND 
ROUTINES THAT ARE APPROPRIATE TO THE INTERFACE EftUIPMENT FEATURES AND 
CAPABILITIES. 

• COMMONALITY** 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE FOR THE USE OF INTERFACE 
STANDARDS FOR PROTOCOLS. ROUTINES. AND DATA REPRESENTATIONS. 


* = New 
** = Different 

























Figure 5 Software Quality Criteria Definitions * = New 


CRITERION 


DEFINITION 


C/l 


•TRACEABILITY 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE A THREAD OF ORIGIN FROM THE 
IMPLEMENTATION TO THE REOUIREMENTS WITH RESPECT TO THE SPECIFIED 
DEVELOPMENT ENVELOPE AND OPERATIONAL ENVIRONMENT. 

•CONSISTENCY 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE FOR UNIFORM DESIGN AND 
IMPLEMENTATION TECHNIOUES AND NOTATION, 

• COMPLETENESS 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE FULL IMPLEMENTATION OF THE 
FUNCTIONS REGUIRED. 

• COMPLIANCE* 

• THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROMOTE IMPLEMENTATIONS THAT 
CONFORM TO THE RERUIREMENTS. 

• VALIDITY* 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH CONSTRAIN IMPLEMENTATIONS TO A 
RANGE OF ACCEPTABLE SOLUTIONS. 

• CLARITY* 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE NON-AMBI GUOUS DESCRIPTIONS 
OF FUNCTIONS AND IMPLEMENTATIONS. 

• SPECIFICITY* 

• THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE SINGULARITY IN THE 
DEFINITION AND IMPLEMENTATION OF FUNCTIONS. 

• SIMPLICITY 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE FOR THE DEFINITION AND 
IMPLEMENTATION OF FUNCTIONS IN THE MOST NON-COMPLEX AND UNDERSTANDABLE 
MANNER. 

• ANOMALY 

MANAGEMENT** 

• THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE FOR CONTINUITY OF 
OPERATIONS UNDER AND RECOVERY FROM NON-NOMINAL CONDITIONS. 

• ACCURACY 

• THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE THE REGUIRED PRECISION IN 
CALCULATIONS AND OUTPUTS. 

• EFFECTIVENESS** 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE FOR MINIMUM UTILIZATION OF 
RESOURCES (PROCESSING TIMEi STORAGEi OPERATOR TIME) IN PERFORMING 
FUNCTIONS. 

• ACCESSIBILITY** 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE FOR CONTROL AND AUDIT OF 
ACCESS TO THE SOFTWARE AND DATA. 

• VIRTUALITY* 

• THOSE ATTRIBUTES OF THE SOFTWARE WHICH PRESENT A SYSTEM THAT DOES NOT 
REOUIRE USER KNOWLEDGE OF THE PHYSICAL CHARACTERISTICS (a.e.. NUMBER OF 
PROCESSORS/DISKSi STORAGE LOCATIONS) 

• VISIBILITY** 

•THOSE ATTRIBUTES OF THE SOFTWARE WHICH PROVIDE STATUS MONITORING OF THE 
DEVELOPMENT AND OPERATION (e.g.i INSTRUMENTATION). 

• COMPREHENSIBILITY* 

• THOSE ATTRIBUTES OF THE SOFTWARE WHICH ENHANCE UNDERSTANDING OF THE 
OPERATION OF THE SOFTWARE. 





























